Clustering of compressed illumination-invariant chromaticity signatures for efficient video summarization

Motivated by colour constancy work in physics-based vision, we develop a new low-dimensional video frame feature that is effectively insensitive to lighting change and apply the feature to keyframe production using hierarchical clustering. The new image feature results from normalising colour channels for frames and then treating 2D histograms of chromaticity as images and compressing these. Because we effectively reduce any video to the same lighting conditions, we can precompute a universal basis on which to project video frame feature vectors. The new feature thus has the advantage of more expressively capturing essential colour information, and is useful for video indexing because it is very low-dimension — the feature vector is only of length 8. We carry out clustering efficiently by adapting the hierarchical clustering data structure to temporally-ordered clusters. Using a new multi-stage hierarchical clustering method, we merge clusters based on the ratio of cluster variance to variance of the parent node, merging only adjacent clusters, and then follow with a second round of clustering. The second stage merges clusters incorrectly split in the first round by the greedy hierarchical algorithm, and as well merges non-adjacent clusters to fuse near-repeat shots. The new summarization method produces a very succinct set of keyframes for videos and, compared to a previous well-known technique, results are excellent.